[torch.compile] Passing only necessary compilation config to inductor pass config#27041
Merged
luccafong merged 5 commits intovllm-project:mainfrom Oct 17, 2025
Merged
Conversation
luccafong
requested review from
ProExpertProg,
youkaichao and
zou3519
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request refactors how compilation configuration is passed to the VllmInductorPass. Instead of passing a weak reference to the entire CompilationConfig, it now passes a new SimplifiedCompilationConfig object containing only the necessary fields. This is a great improvement for safety and decoupling, as it avoids passing potentially dangerous pointers (like static_forward_context) into the torch.compile process. However, I've identified a critical type mismatch in the new SimplifiedCompilationConfig dataclass that needs to be addressed.
luccafong
force-pushed
the
export-D84790018
branch
from
d161999 to
170f61e
Compare
luccafong
force-pushed
the
export-D84790018
branch
from
170f61e to
b8b8c71
Compare
luccafong
force-pushed
the
export-D84790018
branch
from
b8b8c71 to
31ef04c
Compare
luccafong
force-pushed
the
export-D84790018
branch
from
31ef04c to
cdb1bfd
Compare
…lm-project#27041) Summary: Pull Request resolved: vllm-project#27041 we should not pass the weakref to compilation_config, which include static_forward_context that will holds the pointers to the model layers (e.g. moe, attention), which is dangerous, as this will be passed as config to torch.compile Test Plan: local tests Differential Revision: D84790018 Signed-off-by: Lu Fang <fanglu@fb.com>
luccafong
force-pushed
the
export-D84790018
branch
from
cdb1bfd to
948efa4
Compare
houseroad
reviewed
Oct 16, 2025
houseroad
reviewed
Oct 16, 2025
luccafong
force-pushed
the
export-D84790018
branch
from
948efa4 to
5493d3c
Compare
luccafong
force-pushed
the
export-D84790018
branch
from
5493d3c to
1fd5983
Compare
luccafong
changed the title
houseroad
reviewed
Oct 16, 2025
| use_inductor_graph_partition=config.use_inductor_graph_partition, | ||
| compile_sizes=config.compile_sizes, | ||
| ) | ||
| self.pass_config = config.compilation_config.pass_config |
Collaborator
There was a problem hiding this comment.
later pass_config can be also moved.
Collaborator
Author
There was a problem hiding this comment.
I think we can utilize it, but it will introduce duplicated attribute in config level, we can think of how to organize these config better in following PR. @zou3519
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
zou3519
approved these changes
Oct 16, 2025
github-project-automation
Bot
moved this from To triage
to Done
in torch.compile integration
This was referenced Oct 17, 2025
Merged
1 task
ProExpertProg
added this to the
vllm==v0.12.0/torch==2.9.0 compilation improvements milestone
lywa1998
pushed a commit
to lywa1998/vllm
that referenced
this pull request
Oct 20, 2025
… pass config (vllm-project#27041) Signed-off-by: Lu Fang <fanglu@fb.com> Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
alhridoy
pushed a commit
to alhridoy/vllm
that referenced
this pull request
Oct 24, 2025
… pass config (vllm-project#27041) Signed-off-by: Lu Fang <fanglu@fb.com> Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
0xrushi
pushed a commit
to 0xrushi/vllm
that referenced
this pull request
Oct 26, 2025
… pass config (vllm-project#27041) Signed-off-by: Lu Fang <fanglu@fb.com> Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com> Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
0xrushi
pushed a commit
to 0xrushi/vllm
that referenced
this pull request
Oct 26, 2025
… pass config (vllm-project#27041) Signed-off-by: Lu Fang <fanglu@fb.com> Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com> Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
rtourgeman
pushed a commit
to rtourgeman/vllm
that referenced
this pull request
Nov 10, 2025
… pass config (vllm-project#27041) Signed-off-by: Lu Fang <fanglu@fb.com> Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
devpatelio
pushed a commit
to SumanthRH/vllm
that referenced
this pull request
Nov 29, 2025
… pass config (vllm-project#27041) Signed-off-by: Lu Fang <fanglu@fb.com> Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary: we should not pass the weakref to compilation_config, which include static_forward_context that will holds the pointers to the model layers (e.g. moe, attention), which is dangerous, as this will be passed as config to torch.compile
Differential Revision: D84790018